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ABSTRACT 



The primary purpose of the report is to explore the impact of PISA reading scores on the early labour 
market outcomes of young Canadians of the Youth in Transition Survey. This inquiry is complicated hy 
two facts. First, family and school characteristics that are positively correlated with PISA scores are also 
correlated with labour market success, making it difficult to discover the independent effect of those 
scores. Second, students with higher PISA scores are much more likely to pursue education beyond high 
school and scores may operate both directly and indirectly through this channel to influence later 
outcomes. Among females, there is a positive correlation between PISA scores and future earnings, even 
after controlling for family background and educational attainment. There is no evidence of such a 
correlation for males. For both genders, the link between PISA scores and unemployment disappears when 
controls are added. These weak outcomes may be explained by the fact that sufficient time has not elapsed 
for the YITS respondents to complete schooling and to integrate into the labour market. 



RESUME 



F’objectif premier du present rapport est d’examiner comment les scores PISA en comprehension de 
Tecrit influencent les premiers resultats sur le marche du travail des jeunes Canadiens interroges dans le 
cadre de I’Fnquete aupres des jeunes en transition (FJFT). F’analyse est compliquee par deux facteurs : 
tout d’abord, les caracteristiques en termes de contexte familial et d’etablissement scolaire frequente qui 
sont positivement correlees avec les scores PISA sont egalement correlees avec la reussite sur le marche du 
travail, ce qui empeche d’identifier Teffet independant de ces scores. Fnsuite, les eleves qui ont obtenu des 
scores PISA eleves sont beaucoup plus susceptibles de continuer a faire des etudes apres I’enseignement 
secondaire ; les scores pourraient done avoir un effet a la fois direct et indirect sur les resultats ulterieurs. 
Chez les filles, on constate une correlation positive entre les scores PISA et le revenu futur, meme apres 
prise en compte des caracteristiques du contexte familial et du niveau d’etudes. En revanche, on ne dispose 
d’aucun element prouvant une telle correlation chez les hommes. Pour les femmes comme pour les 
hommes, le lien entre les scores PISA et le fait d’etre sans emploi disparait des lors que Ton ajoute des 
controles. Fa faiblesse de ces resultats peut s’expliquer par le manque de temps ecoule entre le moment ou 
les personnes ont ete sondees par TEJET et le moment ou elles ont termine leurs etudes pour entrer sur le 
marche du travail. 
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EXECUTIVE SUMMARY 



The Youth in Transition Survey allows Canadian researchers, for the first time, to examine the 
relationship between the outcomes of high school students on international standardized tests of reading, 
mathematics, and science on labour market outcomes. This paper examines the impact of PISA scores on 
labour market earnings and unemployment, restricting attention to reading scores so as to preserve sample 
size. 



Performance in the PISA tests is highly correlated with educational attainment. Approximately 65 
percent of YITS respondents in the highest score quintile were still in university at the time of the Cycle 4 
survey, with another 13 percent still in college. Over three quarters of those in the lowest quintile had 
finished formal education. This has two important implications for the research. First, in terms of 
understanding the causal impact of higher PISA scores, clearly much of it may operate through an 
educational attainment effect. The second implication is that, in trying to quantify the effect, the inquiry is 
hampered by a lack of data on labour market outcomes of those in the top quintiles who have yet to enter 
the labour market. Moreover, those that have entered the labour market are still very early in their careers 
and may have yet to demonstrate their innate productivity to employers. 

PISA scores have an impact on the earnings of females. With no controls for background or 
educational attainment, a one standard deviation increase in the reading score is estimated to be associated 
with a 5 percent increase in earnings. Adding those controls reduces the estimate to 4.1 percent. The 
impact of the scores for females as a group is driven primarily by the effects these scores have on those 
with completed high school or less, although some care should be exercised in interpreting the outcomes 
for higher levels of education where sample sizes become disconcertingly small. 

The estimated effect of a one standard deviation increase in the reading score for males is only 2.1 
percent and becomes statistically insignificant when background and educational controls are included. 
PISA scores are as strongly correlated with educational attainment for males as they are for females and 
the poor predictive power of PISA scores for earnings for males suggest that they operate only indirectly 
through the attainment channel. 

The impact of PISA scores on the incidence of unemployment is also not particularly large or robust 
to the inclusion of controls. In this case, unemployment incidence is lower for both genders when raw 
comparisons are made but the scores lose their statistical significance for both genders when family 
background and educational attainment controls are introduced. 

The analysis of the effects of performance on the PISA tests has clearly been undertaken too early. 
The fact that a large part of the sample remains in formal schooling has very serious consequences for 
sample sizes. More troubling is the fact that the sample is not randomly reduced by this phenomenon. 
Those with higher scores are more likely to be excluded. Moreover, the personal attributes that lead to 
better performance in the PISA tests will not be immediately evident to employers, who have no 
knowledge of those scores. We need to have at least one more cycle of observation in the YITS to allow 
observation of earnings and unemployment outcomes after individuals from across the entire distribution 
of PISA scores have had time to integrate into the labour market. 
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I. INTRODUCTION 



1. The outcomes of standardised testing in Canadian schools attract considerable attention among 
parents, educators, and policy-makers. That interest derives from an underlying belief that these outcomes 
foretell the economic future of the children being tested. Parents are concerned if the school their child 
attends fares poorly in comparison with others. Any report in the popular press that Canadian children fall 
behind children in other countries in mathematics tests raise alarms about Canada’s longer term ability to 
maintain its relative standard of economic prosperity. 

2. International evidence tends to support the assumption that scores matter, showing significant 
associations between early cognitive skills among children (as measured by test scores) and later labour 
market outcomes.’ Canadian evidence on the issue has not been available to date since data that can link 
cognitive test achievements among children or teenagers and their later labour market success has simply 
not been available.^ In 2000, the Youth in Transition Survey (YITS) administered the OECD’s Programme 
for International Student Assessment (PISA) tests to high school students who were 15 years old in 1999 
and whose education and labour market pathways have been followed since. As this cohort enters the 
labour market, we are now in a position to examine Canadian evidence on the link between test scores and 
success in that market. 

3. This report presents Canadian evidence on the links between cognitive skills in the areas of 
reading and early labour market success of young Canadians.^ It finds that PISA reading scores matter a 
great deal for educational attainment, with those achieving higher scores have a much greater probability 
of proceeding to educational levels beyond high school. However, conditional on that attainment, the 
reading scores generally matter very little for either earnings or employment outcomes. What effects are 
found tend to be among women whose highest educational attainment is completed high school. This 
rather pessimistic result may be tied to the fact that many of the YITS respondents have yet to enter the 
labour market, being engaged in university studies, and those who have left formal schooling have yet to 
establish themselves in that market. 

4. The following section briefly reviews the available literature. Establishing an association between 
success in tests and success in the labour market is not prima facie evidence of causality. Section 3 
discusses the analytical issues involved in establishing a causal link between test scores and outcomes, and 
why this is critical for drawing any policy conclusions. Section 4 describes the YITS data while the 
following section reports the empirical findings. 



1 . See, for example, Currie and Thomas (2001) for British evidence and Murnane et al. (2000) for American 

results. 

2 . The International Adult Literacy Survey report literacy and numeracy skills and labour market outcomes 

but those skills are tested contemporaneously. 

3 . Sample sizes are too small to produce reliable evidence on the impacts of PISA scores in mathematics and 

science. 
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II. LITERATURE REVIEW 



5. Canadian research on the link between ability and labour market outcomes has been limited by 
data availability. Charette and Meng (1998) use the Literacy Skills Used in Daily Activities (LSUDA) data 
to produce the first empirical look at literacy and numeracy impacts in the Canadian context, examining 
labour market outcomes that include earnings, participation and unemployment. Generally, literacy and 
numeracy are found to play significant roles in all of these outcomes. Green and Riddell (2001) confirm 
the importance of literacy in earnings determination using the International Adult Literacy Survey (lALS). 
While the impact of literacy on labour force participation and unemployment is not examined separately, 
earnings are partly determined by the quantity of labour supplied and, in that sense, the impact of literacy 
on these dimensions is captured indirectly. 

6. While these results are informative for the following report, it is important to note that the 
LSUDA and lALS data measure literacy and/or numeracy contemporaneously with labour market 
outcomes. Cognitive skills measured at the same time as labour market success will include the effects of 
post-secondary education and experience to the extent that these contribute to literacy and numeracy skills. 
In the YITS data, on the other hand, the ability measures are constructed well before any higher education 
can be taken or labour market experience gained. Although the PISA scores in the YITS data will be 
influenced by early parental investments in human capital and the quality of the individual’s primary and 
early secondary school experiences, they will be more heavily influenced by innate ability. Moreover, the 
link between these scores and labour market outcomes will be much more complex, involving 
considerations of skills depreciation, confounding of early ability measures and subsequent human capital 
investments, and so on. The YITS data are the first to provide the opportunity to explore the link between 
early cognitive skills and later labour market outcomes and, subsequently, there is as yet no Canadian 
literature on the subject. There is, however, considerable international research. 

7. Mumane et al. (1995) use the American National Longitudinal Study of the High School Class of 
1972 and the High School and Beyond survey of high school seniors in 1980 to examine the link between 
mathematics and reading test scores measured in the senior year of high school and the wages of 24 year 
old men and women who had completed their formal education. Basic cognitive skills, as measured by 
those scores, are found to be significant determinants of wages, more so for the later cohort than the earlier. 
The result is confirmed in Mumane et al. (2000) where, for example, the predicted earnings of a 31 -year- 
old male who graduated from high school with strong cognitive skills are 30 percent higher than one who 
graduated with weak cognitive skills. Much of this difference is explained by the fact that the former is 
much more likely to achieve post-secondary education credentials. Blackburn and Newmark (1993) show 
that cognitive skills matter within educational attainment groups as well, finding that the labour market 
gains from a college education were greater for male high school graduates with stronger cognitive s ki lls. 
Similarly, Tyler et al. (2000) show that cognitive skills matter among young high school dropouts. In a 
widely cited paper, Neal and Johnson (1996) find that differences in the Armed Services Vocational 
Aptitude Battery test scores measured among NLSY panel members in 1980 explain much of the racial 
differences in earnings ten years later. 

8. Currie and Thomas (2001), using data from the British National Child Development Survey of 
children born in 1958, find that age 16 test scores play an important role in wage and employment 
outcomes at age 33. The potential intermediate role of post-secondary educational attainment is not 
explored. Interestingly, Currie and Thomas find that the age 16 test score has a smaller effect on the wages 
and employment probabilities of children from higher socio-economic status families. For example, a one 
standard deviation increase in the math test score at age 16 would translate into a 14 percent higher wage 
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rate at age 33 for individuals from low socio-economic status backgrounds, and all percent wage increase 
for those from high socio-economic status backgrounds. 

9. Evidence using the results of the PISA testing done in the YITS is just now starting to emerge. 
Betshy et al. (2008) used the results of a YITS-like survey in Switzerland to examine the possible links 
between PISA reading scores the quality of the job held five years after the Swiss students completed the 
tests. The research was restricted to those pursuing vocational education, since only this group had 
emerged from formal schooling in sufficient numbers to generate a useful sample size. The authors find 
that PISA scores have no direct effect on the quality of the job but had indirect effects because higher 
scoring students pursued pathways through more demanding vocational streams and these streams 
produced better jobs. In other words, PISA scores did not appear to matter within groups pursuing a 
specific vocational education path. 

10. Turning to the impact of working while in school, American research typically shows that 
working while in school, especially in secondary school, has significant and positive impacts on earnings 
and employment."^ The evidence is not indisputable, however, and Hotz et al. (2002) claim to show that any 
positive impacts of working while in school are statistical artefacts of selection. The most recent Canadian 
evidence is by Parent (2006) who finds, using the 1991 School Leavers Survey and its 1995 Eollowup, that 
working while in high school has no discernible positive impact on subsequent income although there 
appears to be a negative and causal relationship with the probability of graduating from high school. 



4 . See, for example. Light (2001). 
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III. METHODOLOGY 



11. The test scores at the centre of this inquiry are those obtained hy 15 year old Canadian high 
school students in the OECD’s Programme for International Student Assessment (PISA) initiative, which 
establishes an internationally comparative assessment of student outcomes in reading, mathematical and 
scientific literacy. To quote from PISA: 

“The OECD Programme for International Student Assessment (PISA) is a collaborative effort undertaken by 
all member countries and a number of non-member partner countries to measure how well students, at age 15, are 
prepared to meet the challenges they may encounter in future life. Age 1 5 is chosen because at this age, in most 
OECD countries, students are approaching the end of compulsory schooling and so, some measure of the 
knowledge, skills and attitudes accumulated over approximately ten years of education is gained from an 
assessment at this time. The PISA assessment takes a broad approach to assessing knowledge, skills and 
attitudes that reflect current changes in curricula, moving beyond the school based approach towards the use of 
knowledge in everyday tasks and challenges. The skills acquired reflect the ability of students to continue learning 
throughout their lives by applying what they learn in school to non-school environments, evaluating their choices 
and making decisions. (OECD,p. 7). 

12. The language in the OECD quote above is forward looking and suggests that student performance 
in the PISA tests measures their potential for future success. The methodology best suited to addressing the 
question of whether this is, in fact, true requires us to think about the determinants of PISA scores and then 
consider why we might expect them to influence future labour market success. 

13. In and of themselves, PISA scores have no direct currency in the labour market since prospective 
employers are simply not equipped to evaluate them as signals of expected productivity in the same way 
that grade point averages might be used. If measures of cognitive skills at age 15 are to have an effect on 
employment and earnings several years later they must do so through indirect channels. There are two 
potential ways in which PISA scores may be causality and directly related to future labour market success.^ 
Eirst, higher PISA scores may reflect a larger bundle of productivity enhancing skills and knowledge that 
the individual retains and markets after leaving schooling. Second, higher scores may reflect an attitude 
towards learning that portends larger investments in human capital through further education and these 
investments produce success in the labour market. Complicating matters enormously is the possibility that 
additional factors may simultaneously determine both PISA scores and labour market outcomes. Higher 
PISA scores may reflect greater innate ability (as opposed to a bundle of skills and knowledge), greater 
family investments in the child’s development, or school resources. All of these factors continue on after 
the PISA tests are written and may have independent labour market impacts if, for example, families or 
schools invest in activities that improve the transition into the labour market. 

14. Why is this important? The precise mechanism through which PISA scores affect labour market 
outcomes matters very much for policy prescriptions. Suppose, for example, that higher PISA scores affect 
future earnings only because they capture non-cognitive abilities and those abilities are largely the product 



5 . Of course, these are not mutually exclusive and multivariate methods will eventually have to be brought to 

bear to disentangle the independent effects. 
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of the home environment. Investing more money into school resources (for example, lowering the student 
to teacher ratio) would then have no impact on PISA scores or on future labour market success. Suppose, 
on the other hand, that higher scores cause higher future earnings because they measure the cognitive skills 
that high school students have mastered in school. Then increased school resources to improve learning 
would have longer term benefits. Finally, suppose that the linkage from scores to success reflects a 
mechanism whereby high scoring students continue on to PSE and it is the latter human capital process that 
creates productivity. In this case, public policy to increase access to PSE may be as efficient as policies to 
increase PISA scores. 




Figure 1 



15. Inferring the exact nature of the linkages is challenging because an individual’s test outcome and 
his or her labour market success are simultaneously determined by inter-related factors. Figure 1 provides a 
framework for thinking about these confounding influences:^ 

16. To the extent that PISA scores do capture differences in skills acquired at the age of 15 that will 
have labour market value in the future, the association between those scores and labour market outcomes 
operates through the “productivity effects” linkage in Figure 1. Higher scores may also be causally related 
to the level of educational attainment which is conceptualised as further investments in human capital 
(which will eventually be rewarded). The three factors influencing both the student’s PISA score and early 
labour market outcomes are shown in Figure 1 as having influences in both directions. 

17. To deal with this set of inter-related factors, the following methodology is proposed. The 
earnings impact of PISA scores will be estimated using the standard human capital earnings function. That 
function is first estimated using only the PISA score as an explanatory variable in order to capture the 
difference in mean earnings between individuals with different scores. The model is then re-estimated 
using a full set of controls for family background, school resources, and individual characteristics and 



6 . Figure 1 borrows heavily from Rose and Betts (2001), Figure S.6. 
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behaviours. This intermediate model will then determine how much of the original impact of PISA scores 
is attributable to these factors. Finally, educational attainment indicators are introduced to estimate how 
much of the impact of the PISA score remaining in the intermediate model is due to their role in causing 
greater investments in education.^ 

18. The earnings equation is also estimated separately by educational group. To the extent that PISA 
scores have a causal role in determining the final educational attainment of the individual, doing so will 
under-estimate their impact on earnings. Bearing this caveat in mind, however, it is of considerable interest 
to know whether success in PISA tests is important for those who choose not to pursue higher education, 
whether test scores are more important for those who choose university rather than college education, and 
so on. 

19. A similar methodology is used to examine unemployment probabilities. Participation rates are 
sometimes used to gauge labour market success, but are not examined in this paper. For the YITS 
respondents who have finished formal schooling, participation rates are fairly uniform by PISA score and 
high. With little variation in a rate close to 100 percent, PISA scores will have little effect. 



7 . Had sample sizes permitted, it would have been informative to continue with a fourth specification that 

included post-secondary field of study to determine if higher PISA scores produce better labour market 
outcomes by inducing students to enter higher paying but more demanding fields of study. I am grateful to 
a reviewer for pointing this out. 
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IV. DATA 



20. The Youth in Transition Survey is a large, nationally representative longitudinal survey expressly 
designed to examine transitions of Canadian youth through education and into the labour market. Youth 
who were 15 years old on December 31, 1999 were targeted. PISA scores are available only in cohort A of 
the YITS. A reading test was administered to the entire sample, a mathematics test to one half of the 
sample, and a science test to the other half. 

21. To this point, the YITS has gone through 4 cycles every two years. The first cycle was conducted 
in April 2000 to collect information for 1999 as well as background information. In this first cycle, 
information on family background was supplied directly by parents and school administrators were asked 
for detailed information on educational resources available to the YITS individual. A second interview in 
2002 collected information on activities in 2000 and 2001, a third interview in 2004 captured activities 
during 2002 and 2003 and, finally, the last cycle collected information for 2004 and 2005 in the 2006 
interview. Individuals were 15 years old on December 31, 1999 and were 21 years old at the end of 2005. 
The YITS began with 29,330 respondents in cycle 1. The number had fallen to 18,843 in Cycle 4 in 2006. 

22. Educational attainment in the following analysis is assessed as of the end of Cycle 4, i.e., the 
status at December 31, 2005. The following categories were defined: 

• Less than high school. These individuals had not completed their high school certificate 
requirements and were either still participating in secondary education or had left the educational 
system. 

• High School Graduates had received their secondary school diplomas but had not pursued further 
formal schooling at any time since leaving high school. 

• PSE Leavers had entered some form of post-secondary education but had left without receiving 
qualifications and were not engaged in PSE schooling as of the end of 2005. 

• College Graduates had completed the requirements for a college certificate but had not pursued 
university studies. 

• University Graduates had completed undergraduate studies at the Baccalaureate level. 

• College and university students were still engaged in full-time studies at the end of Cycle 4 (age 

21). 

23. Individuals moving from colleges into universities are treated as university students. Eor 
example, an individual completing a college diploma and then pursuing university studies is treated as a 
current university student. 

24. Cycle 1 determined the number of hours worked while attending high school. Weekday and 
weekend hours are combined to determine the average number of hours worked during the week while in 
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high school. For work hours during PSE, the total number of paid hours plus the number of hours working 
on a farm in the most recent year of PSE are used. 

25. Earnings refer to the hourly earnings in the job held in the last 6 months of 2005. If more than 
one job was held, the earnings data refer to the job in which the individual worked the most monthly hours 
during that period. 

26. Means and standard deviations of the primary variables used in the following analysis are 
provided in Appendix Table Al. Note that all estimates in this report are weighted. 
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V. RESULTS 



V.l Which PISA Score? 

27. YITS respondents were assessed in reading, mathematics and science abilities and the data 
therefore offer three potential PISA scores to use in the subsequent analysis. Although the three scores are 
highly correlated, they are not perfectly so. A simple regression of the available math and science scores, 
respectively, on the reading score shows, through the coefficient of determination, the degree of 
correlation. 



PISA Science Score — 164.02 + 0.683 Heading Score: — 0.32 

( 0 . 012 ) 

PISA Math. Score —21 1.20 + 0.599 Reading Score: R^ — 0.43 

( 0 . 012 ) 



28. It would be informative to understand how each of these scores affects labour market outcomes 
but the issue of sample size will become critical and the need to retain the full YITS sample by using only 
the reading score trumps the desire to look at each score individually.^ 

29. It is important to recognise that PISA scores differ significantly between males and females. For 
females, the mean and standard deviation of the reading score are 549.2 and 88.2, respectively. The 
corresponding values for males are 518.8 and 96.2. Were we to look simply at the correlation between 
PISA scores and labour market earnings using pooled data, the continuing male advantage in earnings will 
produce an underestimate of the impact of the scores. In the following, estimates are therefore produced 
separately for males and females. 



8 . The power of each score to predict PSE attendance was compared by estimating a simple probit model of 

PSE attendance against each score separately. Eor both males and females, the reading score is a better 
predictor than either the math or the science scores according to the pseudo-R^ statistic. It would be 
interesting to examine the influences of the three scores separately, but the sample sizes would be too small 
to produce robust results until more of the sample has had time to enter the labour market. Note, as well, 
that most of the action from PISA scores will be found to work through educational attainment so that use 
of the score best able to predict that attainment, the reading score, would be the preferred option in any 
case. 
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V.2 What are the Determinants of PISA Scores? 

Table 1 . Determinants of PISA Scores 



Determinant 


Males 


Females 


Mother’s Education (ref. group: HS or iess) 






Some Postsecondary: Incomplete 


6.86 


9.02 


College 


17.03** 


20.85** 


University 


39.57** 


38.38** 


Father’s Education (ref. group: HS or less) 






Some Postsecondary: Incomplete 


6.57 


15.25* 


College 


16.29** 


12.07** 


University 


35.92** 


36.22** 


Parent’s Combined Income ($,000’s) 


1.14** 


0.97** 


Province: (ref. group: Ontario) 






NFLD 


-20.93** 


-7.59 


PEI 


-26.17** 


-16.33** 


NS 


-16.91** 


-4.06 


NB 


-14.05* 


-2.65 


QUE 


3.39 


12.73** 


MAN 


3.07 


14.19** 


SASK 


-4.42 


7.36 


ALTA 


14.18** 


20.71** 


BC 


10.65* 


10.55** 


School’s Physical Infrastructure Index (SCMATBUI) 


1.87 


-1.12 


School’s Educational Resources Index (SCMATEDU) 


-4.04* 


-1.16 


Student Teaching Staff Ratio 


1.19* 


1 .09** 


First Generation Immigrant 


-32.00** 


-23.97** 


Second Generation Immigrant 


-0.13 


2.23 


Anglophone in Quebec 


13.23* 


2.82 


Francophone Outside Quebec 


-48.60** 


-37.93** 


Constant 


478.20** 


504.06** 


R“^ 


0.13 


0.12 


No. of Obs. 


7154 


7543 



* indicates significance at 5%, ** indicates significance at 1% 



30. To what extent do parental characteristics and school inputs influence PISA scores? In other 
words, what are the influences running from right to left in the diagram above and is there any scope for 
policy intervention (through, for example, increased school resources). As shown in Table 1, socio- 
economic background plays a very significant role in determining PISA scores. Having parents with 
completed PSE is associated with a large increase in the PISA score and family income has an additional 
positive effect. 

31. The evidence on the role of school resources is more mixed. The state of physical and 
educational infrastructure appears to play no role among females and, in fact, the level of educational 
infrastructure has a perverse sign in the case of males. Higher staff-to-student ratios do seem to increase 
PISA scores for both genders. To the extent that school resources are homogeneous within provinces, some 
of the explanatory power of these variables may be lost through the use of provincial indicator variables 
which show a very distinct east-to-west geographical pattern, particularly for males. Provinces differ in 
their per capita expenditure on elementary and secondary schools with a similar geographic pattern, i.e., 
lower expenditures in the east and higher expenditures in the west.^ If public funding of schools is at the 
root of the provincial differences in PISA scores and if, in turn, we find that higher PISA scores produce 
better labour market outcomes, the scope for policy intervention is important and obvious. 



9 . See, for example, Guillemette (2005), Table 1. 
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32. First generation immigrants performed relatively poorly in the PISA reading tests, although there 
are no significant differences for second generation immigrant students. Clearly, the inference is that first 
generation immigrant children are disadvantaged in reading tests hy language difficulties. A similar effect 
may explain the very poor performance on the PISA test hy francophone children in some provinces 
outside Quebec who, unlike Anglophones inside that province, may not have access to schooling in their 
mother tongue. 

33. What can we conclude from Table 1? First, there appears to be some scope for policy 
intervention that will increase PISA scores, although that scope is limited. If the pattern of provincial 
differences does represent provincial patterns of funding for primary and secondary education, then student 
performance in reading tests can be boosted through increases in funding. The second lesson is that PISA 
scores are heavily influenced by family background. Since factors such as parental education and family 
income are generally found to be important influences on educational attainment and labour market 
success as well, they must be controlled for in the wage and unemployment regressions below. 

V.3 PISA Scores and Educational Attainment 

34. One channel through which PISA scores may influence eventual labour market success is a 
causal influence on educational attainment. Before turning to an examination of the impact of those scores 
on earnings and employment, an examination of the educational patterns by score level is informative. 

35. The YITS sample was arranged by score quintile and, within each quintile, the population was 
apportioned among various possible educational states in Cycle 4 or 2006. Those states are: incomplete 
high school, high school graduation only, PSF leaving without certification (college and university 
combined), college graduation, university graduation, continuers in college, and continuers in university. 
The results are reported in a contingency table format in Table 2. 

Table 2. PISA Scores and Educational Attainment 



PISA 

Quintile 


HS 

Dropout 


HS 

Graduate 


PSE 

Dropout 


College 

Graduate 


University 

Graduate 


College 

Continuer 


University 

Continuer 


1 


0.189 


0.375 


0.082 


0.117 


0.003 


0.151 


0.084 


2 


0.092 


0.267 


0.116 


0.148 


0.005 


0.154 


0.219 


3 


0.044 


0.198 


0.121 


0.123 


0.004 


0.153 


0.358 


4 


0.023 


0.152 


0.101 


0.109 


0.012 


0.107 


0.497 


5 


0.014 


0.087 


0.082 


0.077 


0.011 


0.074 


0.655 


Total 


0.073 


0.216 


0.100 


0.115 


0.007 


0.128 


0.361 



36. There is a very clear and significant correlation between the PISA score and educational 
attainment. Among those in the lowest quintile, over 56 percent had not proceeded beyond high school by 
the age of 21. About 8 percent unsuccessfully tried PSF, 27 percent either graduated from college or are 
continuing a college education, and only 8 percent are university graduates or continuers. By contrast, 90 
percent of those in the highest quintile proceeded to PSE. Interestingly, the proportion who had entered 
college or university but dropped out without completing is the same in the top and the bottom quintiles. 
College becomes a less attractive destination moving up through the score distribution and the increase in 
PSE participation is explained by the dramatic increase in university participation. Eess than 9 percent of 



17 




EDU/WKP(2010)2 



those in the lowest score quintile are either undertaking university studies or have competed those studies. 
By contrast, within the highest quintile, almost 67 percent are university graduates or continuers. 

37. Table 2 does not establish causality. It may be that third factors such as parental income or 
education cause both higher PISA scores and higher PSE participation. Nevertheless, it is clear that caution 
must be exercised if the impacts of PISA scores on earnings are taken in isolation within educational 
categories. Eor example, simply observing a correlation between scores and earnings among high school 
graduates would under-estimate the impact of those scores on earnings by ignoring the fact that higher 
scores are associated with higher educational attainment which, in turn, increases earnings. 

V.4 PISA Scores and Earnings 

38. Before any multivariate analysis is done. Table 3 tabulates population weighted mean earnings 
for males and females by PISA quintile in the YITS data. Earnings are hourly wage rates in jobs held by 
respondents at the end of the cycle. The sample is restricted to those holding jobs at that time. Even in raw 
dollar amounts, there is surprisingly little variation in hourly earnings by quintile. There is some 
progression for females, with the average wage for those in the fifth quintile about 13 percent higher than 
the average in the lowest quintile. Eor males, however, the top quintile earnings are less than 6 percent 
higher than the lowest quintile and the increase moving up through the quintiles is not monotonic, with the 
second quintile receiving the highest hourly earnings. It will not be surprising, then, when the PISA score 
is found to have little impact on earnings in the multivariate analysis to follow. 

Table 3. PISA Scores and Hourly Earnings 



Quintile 


Males 


Females 


First 


$12.97 


$10.30 


Second 


14.10 


10.90 


Third 


13.57 


11.20 


Fourth 


13.87 


11.28 


Fifth 


13.73 


11.65 



39. As described above, the appropriate methodology for investigating the implications of PISA 
reading scores on earnings is to conduct a series of earnings regressions, beginning with a simple model of 
earnings against the score, then adding control variables like family background, and finally accounting for 
the level of educational attainment. There are several important limitations in the YITS data when 
investigating earnings impacts of PISA scores. Many YITS respondents were continuing their education 
during 2006 and the sample needs to be restricted to those who have completed formal education and have 
entered the labour force. Almost half of the sample is lost. This is not only a matter of losing degrees of 
freedom in the econometrics to follow. The YITS respondents still in school are predominantly university 
students and the high correlation between educational attainment and PISA scores means that the missing 
observations tend to come from the upper end of the distribution of those scores. In looking at the impact 
of PISA scores, then, there is a limited ability to observe those with high scores. Another important 
statistical issue arises due to the very short period of time graduates from PSE, especially university 



10 . Note the very small percentage of all quintiles (0.69%) who are university graduates by the time of the 
Cycle 4 interview. This will cause serious sample size issues in later estimates. 
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graduates, have had to establish themselves in the labour market.'* Many graduates of higher education 
face labour market entry difficulties and show early levels of both earnings and employment that grossly 
under-estimate their eventual labour market success.*^ Comparing the earnings of high school graduates 
with relatively low PISA scores but with 4 or 5 years of labour market experience to the earnings of 
university graduates with relatively high PISA scores but only a few months of labour market experience 
will clearly tend to bias the case against any positive contribution of those scores to earnings. The 
estimates with controls do include measures of the time elapsed since graduation but no controls will 
overcome the fundamental problem here which is that we do not have enough information to know how 
PSE graduates will do in the labour force. Einally, as has already been noted, 2006 data do not contain a 
large enough sample size for university graduates to produce statistically reliable estimates for them. This 
inquiry may be taking place too early to fully understand the impact of PISA scores on the labour market 
outcomes of PSE graduates. 

40. Table 4A provides the estimates of earnings impacts for females, where the natural logarithm of 
earnings is used as the dependent variable. Recall that the PISA score measure has been standardised 
within each gender to permit easier interpretation so that a unit change in this dependent variable is a one 
standard deviation change in the raw PISA score. Thus, for someone with an average score, a one standard 
deviation increase in the score will move the individual from the 50* percentile to approximately the 85* 
percentile. Recall, as well, that earnings are wages in the job with the highest number of monthly hours in 
the last 6 months of 2005. Einally, note many observations in the data are missing values for one or more 
variables and must be dropped from the estimation, further reducing sample size. To allow comparability, 
the sample used in all three specifications is that which is available for the most expansive specification. 



11. In the literature, researchers typically use samples of individuals who have had sufficient time to establish 
themselves in the labour market. For example, Currie and Thomas (2001) use labour market data on 33 
year olds while Murnane et al. (2000) use earnings data for individuals 10 or 12 years after high school. 

12 . See, for example, Drewes and Giles (2002). 
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Table 4A PISA Scores and Hourly Earning - Females 



Dependent Variabie: Log of Earnings 


No Controls 


Background 

Controls 


Educational 

Controls 


PiSA Score 


0.050** 


0.048** 


0.041* 


Parent's Combined income ($,000's) 




0.004 


0.003 


Mother’s Education (ref. group: HS or iess) 








Some Postsecondary: Incompiete 




0.089 


0.088 


Coilege 




0.009 


0.006 


University 




0.021 


0.026 


Father's Education (ref. group: HS or iess) 








Some Postsecondary: incompiete 




-0.102 


-0.112 


Coilege 




-0.005 


-0.009 


University 




0.015 


0.021 


School’s Physical Infrastructure Index 




0.002 


0.005 


School’s Educational Resources Index 




0.009 


0.009 


Student Teaching Staff Ratio 




0.003 


0.002 


Time in Labour Force (months) 




-0.000 


0.002* 


Tenure in Current Job (months) 




-0.001 


-0.001 


First Generation Immigrant 




0.079 


0.084 


1®* Gen. Immigrant ’* PISA Score 




0.069 


0.055 


Second Generation Immigrant 




-0.035 


-0.039 


2 ’"^ Gen. Immigrant PISA Score 




0.002 


0.007 


Anglophone in Quebec 




-0.064 


-0.070 


Anglophone * PISA Score 




-0.057 


-0.076 


Francophone Outside Quebec 




0.032 


0.003 


Francophone * PISA Score 




-0.058 


-0.060 


Hrs. Worked While in High School 




0.002 


0.001 


HS Hours’*PISA Score 




-0.000 


-0.000 


Hrs. Worked While in PSE 




0.003** 


0.002 


PSE Hrs.*PISA Score 




-0.002 


-0.002 


Province of job: (ref. group: Ontario) 








NFLD 




-0.239** 


-0.245** 


PEI 




-0.156** 


-0.161** 


NS 




-0.169** 


-0.166** 


NB 




-0.107** 


-0.109** 


QUE 




0.010 


0.028 


MAN 




-0.004 


0.005 


SASK 




-0.033 


-0.030 


ALTA 




0.068 


0.071* 


BC 




-0.019 


-0.017 


Constant 


2.353** 


2.258** 


2.101** 


Educational Attainment (ref. group: <HS) 








HS grad 






0.077* 


College Leaver 






0.054 


University Leaver 






0.050 


College Graduate 






0.219** 


University Graduate 






0.157 


R" 


0.02 


0.07 


0.10 


No. of Obs. 


2380 


2380 


2380 



* indicates significance at 5%, ** indicates significance at 1% 
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Table 4B. PISA Scores and Hourly Earnings - Males 



Dependent Variable: Log of Earnings 


No Controls 


Background 

Controls 


Educational 

Controls 


PISA Score 


0.021* 


0.018 


0.008 


Parent's Combined Income ($,000’s) 




0.003 


0.003 


Mother’s Education (ref. group: HS or less) 








Some Postsecondary: Incomplete 




-0.026 


-0.028 


College 




0.027 


0.022 


University 




-0.017 


-0.024 


Father’s Education (ref. group: HS or less) 








Some Postsecondary: Incomplete 




-0.016 


-0.021 


College 




-0.026 


-0.031 


University 




-0.017 


-0.013 


School’s Physical Infrastructure Index 




0.018 


0.014 


School’s Educational Resources Index 




-0.011 


-0.011 


Student Teaching Staff Ratio 




0.003 


0.003 


Time in Labour Force (months) 




0.002** 


0.003** 


Tenure in Current Job (months) 




-0.001 


0.000 


First Generation Immigrant 




-0.076 


-0.068 


P' Gen. Immigrant * PISA Score 




-0.016 


0.001 


Second Generation Immigrant 




0.014 


0.012 


2"“^ Gen. Immigrant * PISA Score 




-0.024 


-0.017 


Anglophone in Quebec 




-0.133* 


-0.137* 


Anglophone * PISA Score 




-0.044 


-0.042 


Francophone Outside Quebec 




0.079 


0.064 


Francophone * PISA Score 




0.040 


0.036 


Hrs. Worked While in High School 




0.022** 


0.022** 


HS Hours’*PISA Score 




-0.001 


-0.001 


Hrs. Worked While in PSE 




0.003* 


0.002 


PSE Hrs.*PISA Score 




0.000 


0.001 


Province of job: (ref. group: Ontario) 








NFLD 




-0.205** 


-0.213** 


PEI 




-0.191** 


-0.200** 


NS 




-0.155** 


-0.152** 


NB 




-0.173** 


-0.174** 


QUE 




-0.007 


0.011 


MAN 




-0.096** 


-0.086** 


SASK 




-0.022 


-0.031 


ALTA 




0.212** 


0.214** 


BC 




0.094** 


0.091** 


Constant 


2.542** 


2.361** 


2.235** 


Educational Attainment (ref. group: < HS) 








HS grad 






0.079** 


College Leaver 






0.062 


University Leaver 






0.026 


College Graduate 






0.178** 


University Graduate 






0.187 


R2 


0.003 


0.11 


0.12 


No. of Obs. 


2988 


2988 


2988 



* indicates significance at 5%, ** indicates significance at 1 % 



41. In the parsimonious regression, the net impact of a one standard deviation increase in an 
individual’s PISA score relative to other females is a statistically significant increase of 5 percent in 
earnings. Adding the set of family background, school resources, and demographic controls results in only 
a very minor reduction in the estimated impact. To determine whether the PISA score impact depended on 
immigrant, francophone outside Quebec, Anglophone inside Quebec statuses, these indicators were 
interacted with the PISA reading score deviation. The coefficient on the PISA score then becomes the 
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impact of a one standard deviation increase in the reading score for an individual who is not an immigrant, 
not a francophone outside Quebec, not an Anglophone inside Quebec, and who did not work while in 
school. The coefficients on the interactive terms then are interpreted as the difference in the impact of the 
PISA score on earnings for an individual of each one of those attributes except for hours of work in which 
case the coefficient on the interaction term allows the impact of the score to vary continuously with hours 
worked. None of these interactions were significant, suggesting that the impact of the score was uniform 
across these groups. Indeed, remarkably few of the control variables are significant in the earnings 
equation. Aside from the provincial indicators, only a measure of work while attending PSE achieves a 
significance level of at least 5 percent. Erom the estimates, working while in college or university appears 
to pay a small dividend in terms of future earnings. Augmenting the equation with indicators of educational 
attainment further reduces the estimated effect of the PISA score but it remains positive and significant 
nonetheless. Compared to those females who did not complete high school, high school graduates earn 
about 8 percent more and college graduates earn approximately 22 percent more. Incomplete college or 
university studies produce no gains over incomplete high school. The lack of significance on the indicator 
for university graduation in all likelihood is a result of the small sample size for this group. 

42. Taking the three specifications together indicates that PISA scores have a significant effect on 
earnings for women that is independent of their backgrounds and works primarily within educational 
classes. As will be seen below, these effects result primarily from a positive impact on earnings for women 
who have less than completed high school and women who have only a high school certificate. 

43. The results are substantially different for males (see Table 4B). The PISA score is statistically 
significant only in the first specification and, even in this specification, is less than half the effect found for 
females. Once background controls are introduced into the specification, test scores are found to have no 
independent effects. This suggests that the effects found in the first specification actually result from the 
PISA score serving as a proxy for earnings related background characteristics. Of course, given the lack of 
any significant effect in the model with controls, one would not expect the addition of educational 
attainment to be of any interest. If males are not rewarded for higher PISA scores, it makes no sense to ask 
if the rewards occur within educational classes or are the result of greater educational attainment for those 
with higher scores.’^ 

V.5 PISA Scores and Earnings Within Educational Groups 

44. As already stated, looking for effects of PISA scores on earnings within educational groups will 
bias the estimates. Suppose that higher scores are causally linked to a higher probability of pursuing post- 
secondary education. Restricting our attention to, say, high school graduates will miss that part of the effect 
of higher scores associated with greater educational attainment. As well, if the sample is restricted to only 
those who achieve, at best, a high school diploma. Table 2 has already shown that there is a very strong 
correlation between the PISA score and the probability of pursuing PSE. The use of this group to 
investigate the effect of the score on earnings, requires the use of observations on individuals whose score 
is in the upper quintiles but who, for some reason, have chosen not to go on to PSE. If there is an 
unobserved factor, such as a lack of ambition, that is negatively correlated with both labour earnings and 
the probability of pursuing higher education then it may well be that individuals in the higher score 
quintiles suffer more from a lack of ambition than those in the lower quintiles (whose failure to go on in 
education may be due to lack of ability, not ambition). On this account (which is a selectivity story), the 
impact of PISA scores would be underestimated again. 



13 . Note that this does not suggest higher educational attainment does not matter for males. In Table 4B, all 
coefficients on educational attainment relative to incomplete high school are positive (although not all are 
statistically significant), indicating gains to males for completing high school and for attending PSE. 
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45. Nevertheless, examining the impact of PISA scores (if any) within educational attainment classes 
sheds some light on the results of Tables 4A and 4B. Tables 5 through 8 report the earnings equation 
estimates for those without completed high school, those with only high school graduation, PSE dropouts 
(college and university combined), and college graduates. Estimates for university graduates were tried but 
the sample sizes (69 females and 38 males) were too small to produce reliable results. 

Table 5. PISA Scores and Hourly Earnings - Less Thank High School 



Dependent Variabie: Log of Earnings 


Females 


Males 


PiSA Score 


0.149** 


0.047 


Parent's Combined income ($,000's) 


0.100 


0.058 


Mother’s Education (ref. group: HS or iess) 






Some Postsecondary: Incompiete 


0.017 


0.003 


Coilege 


-0.047 


-0.037 


University 


-0.127 


-0.114 


Father's Education (ref. group: HS or iess) 






Some Postsecondary: incompiete 


-0.370** 


-0.205 


Coilege 


-0.019 


-0.156** 


University 


-0.023 


-0.142 


Schooi’s Physicai Infrastructure Index 


-0.045 


0.075* 


School’s Educational Resources Index 


-0.015 


-0.021 


Student Teaching Staff Ratio 


0.008 


-0.005 


Time in Labour Force (months) 


0.006** 


0.005** 


Tenure in Current Job (months) 


0.004* 


-0.001 


First Generation Immigrant 


3.266** 


-0.298** 


P* Gen. Immigrant * PISA Score 


2.125** 


0.037 


Second Generation Immigrant 


-0.475** 


0.516 


2"*^ Gen. Immigrant * PISA Score 


-0.305** 


0.393 


Anglophone in Quebec 


-0.238 


-0.541* 


Anglophone * PISA Score 


-0.281* 


-0.315 


Francophone Outside Quebec 


0.356 


-0.060 


Francophone * PISA Score 


0.217 


0.027 


Hrs. Worked While in High School 


-0.004 


-0.002 


HS Hours*PISA Score 


-0.010* 


-0.003 


Province of job: (ref. group: Ontario) 






NFLD 


-0.524** 


0.019 


PEI 


-0.567** 


-0.077 


NS 


-0.347* 


-0.192** 


NB 


-0.219 


-0.202* 


QUE 


-0.085 


-0.062 


MAN 


-0.030 


-0.087 


SASK 


-0.129 


-0.189* 


ALTA 


-0.206 


0.234** 


BC 


-0.114 


0.068 


Constant 


2.026** 


2.459** 


R" 


0.42 


0.28 


No. of Obs. 


135 


367 



* indicates significance at 5%, ** indicates significance at 1% 
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Table 6. PISA Scores and Hourly Earnings - High Schooi Grads 



Dependent Variabie: Log of Earnings 


Femaies 


Maies 


PiSA Score 


0.062** 


0.010 


Parent's Combined income ($,000's) 


0.010 


0.047 


Mother’s Education (ref. group: HS or iess) 






Some Postsecondary: Incompiete 


0.140 


-0.008 


Coilege 


0.006 


0.020 


University 


-0.150* 


-0.039 


Father's Education (ref. group: HS or iess) 






Some Postsecondary: incompiete 


-0.103 


0.016 


Coilege 


0.042 


0.026 


University 


0.110 


0.045 


Schooi’s Physicai Infrastructure Index 


0.034 


-0.004 


School’s Educational Resources Index 


-0.005 


-0.012 


Student Teaching Staff Ratio 


0.005 


0.009* 


Time in Labour Force (months) 


0.000 


0.002* 


Tenure in Current Job (months) 


-0.001 


0.000 


First Generation Immigrant 


0.039 


-0.121 


1®* Gen. Immigrant * PISA Score 


0.077 


-0.007 


Second Generation immigrant 


-0.073 


-0.147* 


2"“^ Gen. Immigrant * PiSA Score 


-0.067 


-0.075 


Angiophone in Quebec 


-0.147 


-0.160* 


Angiophone * PISA Score 


0.033 


-0.063 


Francophone Outside Quebec 


0.111 


-0.041 


Francophone * PISA Score 


-0.053 


-0.016 


Hrs. Worked While in High Schooi 


0.001 


0.002* 


HS Hours*PISA Score 


-0.001 


-0.001 


Province of job: (ref. group: Ontario) 






NFLD 


-0.282** 


-0.294** 


PEi 


-0.179** 


-0.276** 


NS 


-0.102 


-0.160** 


NB 


-0.142** 


-0.223** 


QUE 


-0.016 


0.017 


MAN 


-0.019 


-0.080 


SASK 


-0.034 


-0.067 


ALTA 


0.090 


0.190** 


BC 


-0.025 


0.057 


Constant 


2.181 


2.236** 


R" 


0.12 


0.14 


No. of Obs. 


913 


1393 



* indicates significance at 5%, ** indicates significance at 1% 
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Table 7. PISA Scores and Hourly Earnings - PSE Leavers 



Dependent Variabie: Log of Earnings 


Females 


Males 


PiSA Score 


0.040 


-0.040 


Parent's Combined income ($,000's) 


-0.013 


0.029 


Mother’s Education (ref. group: HS or iess) 






Some Postsecondary: Incompiete 


0.002 


-0.055 


Coilege 


0.018 


0.046 


University 


0.120 


-0.004 


Father's Education (ref. group: HS or iess) 






Some Postsecondary: incompiete 


0.051 


-0.098 


Coilege 


-0.094 


-0.075 


University 


-0.164 


-0.025 


Schooi’s Physicai Infrastructure Index 


-0.012 


-0.020 


School’s Educational Resources Index 


0.034 


-0.003 


Student Teaching Staff Ratio 


-0.001 


0.009 


Time in Labour Force (months) 


0.006** 


0.004* 


Tenure in Current Job (months) 


-0.000 


0.001 


First Generation Immigrant 


-0.172 


0.012 


1®* Gen. Immigrant * PISA Score 


-0.048 


-0.246 


Second Generation Immigrant 


0.022 


0.102 


2"“^ Gen. Immigrant * PISA Score 


0.110 


-0.265 


Anglophone in Quebec 


-0.091 


-0.118 


Anglophone * PISA Score 


-0.045 


-0.082 


Francophone Outside Quebec 


-0.049 


0.101 


Francophone * PISA Score 


0.019 


0.035 


Hrs. Worked While in High School 


0.002 


0.003 


HS Hours*PISA Score 


0.000 


-0.001 


Hrs. Worked While in PSE 


0.006** 


0.004 


PSE Hrs.*PISA Score 


-0.002 


0.001 


Province of job: (ref. group: Ontario) 






NFLD 


-0.151 


-0.241* 


PEI 


-0.110 


-0.180* 


NS 


-0.105 


-0.220** 


NB 


-0.019 


-0.111 


QUE 


0.041 


0.001 


MAN 


0.069 


-0.110 


SASK 


-0.032 


0.032 


ALTA 


-0.022 


0.110 


BC 


0.080 


0.175* 


Constant 


2.087 


2.14** 


R" 


0.15 


0.15 


No. of Obs. 


480 


561 



* indicates significance at 5%, ** indicates significance at 1% 
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Table 8. PISA Scores and Hourly Earnings - College Grads 



Dependent Variable: Log of Earnings 


Females 


Males 


PISA Score 


0.019 


0.007 


Parent's Combined Income ($,000's) 


-0.017 


-0.059 


Mether’s Educatien (ref. group: HS or less) 






Some Postsecendary: Incomplete 


0.097 


-0.023 


College 


-0.033 


0.034 


University 


0.093* 


-0.038 


Father's Educatien (ref. group: HS or less) 






Some Postseccndary: Incomplete 


-0.171 


0.093 


College 


-0.031 


-0.010 


University 


0.063 


-0.009 


School’s Physical Infrastructure Index 


-0.043 


0.042 


School’s Educational Resources Index 


0.021 


-0.024 


Student Teaching Staff Ratio 


-0.003 


-0.006 


Time in Labeur Force (months) 


-0.003 


0.004* 


Tenure in Current Jeb (menths) 


-0.001 


-0.000 


First Generation Immigrant 


0.084 


0.106 


P* Gen. Immigrant * PISA Score 


0.004 


0.157* 


Second Generation Immigrant 


-0.079 


0.121 


2"“^ Gen. Immigrant * PISA Scere 


0.141 


0.064 


Anglophone in Quebec 


-0.054 


-0.036 


Angicphone * PISA Scere 


-0.154 


-0.140* 


Francophone Outside Quebec 


-0.083 


0.199* 


Francophone * PISA Scere 


-0.106 


0.026 


Hrs. Wcrked While in High School 


0.003 


0.002 


HS Hours*PISA Score 


0.000 


-0.001 


Hrs. Wcrked While in PSE 


-0.002 


0.001 


PSE Hrs.*PISA Score 


-0.002 


0.000 


Province cf jeb: (ref. group: Ontario) 






NFLD 


-0.284** 


-0.172* 


PEI 


-0.179** 


-0.179* 


NS 


-0.255** 


-0.113 


NB 


-0.123* 


-0.146* 


QUE 


0.053 


0.010 


MAN 


-0.193 


-0.122 


SASK 


0.030 


0.029 


ALTA 


0.152** 


0.245** 


BC 


-0.086 


0.123 


Constant 


2.543** 


2.593 


R" 


0.11 


0.13 


No. of Obs. 


783 


629 



46. Table 5 reports the results for females and males without completed high school. The PISA score 
is insignificant for the earnings of these males but is both significant and numerically very large for 
women. Some caution should be exercised in interpreting the coefficients in this regression, given the 
small sample size. The results for first and second generation immigrant women reflect a small but 
distinctly unrepresentative number of immigrant women within this educational class. 

47. The larger sample sizes for high school graduates provide a higher level of comfort with the 
results and the pattern continues for this group. A standard deviation increase in the PISA score contributes 
over 6 percent to the earnings of females but has no significant effect on the earnings of males. Aside from 
the expected pattern of differences between provinces, very few of the explanatory variables are 
significant. 
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48. Moving to PSE leavers and college graduates leaves statistical significance for the PISA score 
behind for females. The positive impact of higher test scores on earnings for women observed in Table 4A 
may then be interpreted as coming largely from gains made for women with completed high school or less, 
with only a small contribution from the additional education undertaken by those with higher scores. 

V.6 PISA Scores and Unemployment 

49. Unemployment is captured by a positive response to the question of whether any job search 
activity had taken place in the final 6 months of Cycle 4. The overall incidence of this measure was 9.3 
percent for females and 12.0 percent for males and the incidence by PISA quintile is provided in the 
tabulations in Table 9. The relationship between the scores and the incidence of unemployment is 
considerably stronger than appears to be the case for PISA scores and earnings. 

Table 9. PISA Scores and Unemployment 



Quintile 


Males 


Females 


First 


1 3.9% 


1 1 .9% 


Second 


12.9 


8.4 


Third 


10.4 


10.1 


Fourth 


10.1 


6.1 


Fifth 


7.9 


9.0 



50. Tables lOA and lOB report the results of probit regressions of the incidence of unemployment, 
using the same procedure as used for the earnings regressions, i.e., beginning with a simple probit model 
where the PISA score is the explanatory variable, incorporating a set of controls, and finally introducing 
educational attainment indicators. 
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Table 10A. PISA Scores and Unemployment - Females Probit Estimates of Marginal Effect 



Dependent Variabie: Binary Indicator for 
Unempioyment 


No Controls 


Background 

Controls 


Educational 

Controls 


PiSA Score 


-0.019* 


-0.017 


-0.012 


Parent’s Combined income ($,000's) 




-0.042 


-0.042 


Mother’s Education (ref. group: HS or iess) 








Some Postsecondary: Incompiete 




-0.033 


-0.033 


Coilege 




0.005 


0.006 


University 




-0.002 


-0.008 


Father’s Education (ref. group: HS or iess) 








Some Postsecondary: incompiete 




0.016 


0.017 


Coilege 




0.005 


0.006 


University 




0.063 


0.059 


Schooi’s Physicai Infrastructure Index 




0.021 


0.021 


School’s Educational Resources Index 




-0.027** 


-0.027** 


Student Teaching Staff Ratio 




-0.003 


-0.001 


Time in Labour Force (months) 




-0.001 


-0.001 


First Generation Immigrant 




0.081 


0.081 


P* Gen. Immigrant * PISA Score 




-0.019 


-0.017 


Second Generation Immigrant 




0.034 


0.039 


2"“^ Gen. Immigrant PISA Score 




0.015 


0.014 


Anglophone in Quebec 




0.032 


0.036 


Anglophone ’* PISA Score 




-0.024 


-0.019 


Francophone Outside Quebec 




-0.051 


0.059 


Francophone * PISA Score 




0.075** 


0.074** 


Hrs. Worked While in High School 




-0.000 


-0.000 


HS Hours*PISA Score 




-0.000 


-0.000 


Hrs. Worked While in PSE 




-0.001 


-0.001 


PSE Hrs.*PISA Score 




0.001 


0.001 


Province (ref. group: Ontario) 








NFLD 




0.081* 


0.082* 


PEI 




0.087* 


0.096* 


NS 




0.000 


-0.001 


NB 




0.001 


0.010 


QUE 




-0.005 


-0.004 


MAN 




-0.021 


-0.021 


SASK 




0.016 


0.018 


ALTA 




-0.002 


0.001 


BC 




0.028 


0.032 


Educational Attainment (ref. group: < HS) 








HS grad 






-0.039 


College Leaver 






-0.020 


University Leaver 






-0.000 


College Graduate 






-0.036 


University Graduate 






-0.014 


Pseudo R^ 


0.006 


0.05 


0.05 


No. of Obs. 


3198 


3198 


3198 



* indicates significance at 5%, ** indicates significance at 1% 
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Table 10B. PISA Scores and Unemployment - Males Probit Estimates of Marginal Effects 



Dependent Variable: Binary Indicator for 
Unemployment 


No Controls 


Background 

Controls 


Educational 

Controls 


PISA Score 


-0.025** 


-0.018 


-0.020 


Parent’s Combined Income ($,000's) 




-0.017 


-0.018 


Mother’s Education (ref. group: HS or less) 








Some Postsecondary: Incomplete 




0.038 


0.034 


College 




-0.001 


0.000 


University 




-0.008 


-0.009 


Father’s Education (ref. group: HS or less) 








Some Postsecondary: Incomplete 




0.036 


0.037 


College 




0.009 


0.006 


University 




-0.034 


-0.036 


School’s Physical Infrastructure Index 




-0.011 


-0.011 


School’s Educational Resources Index 




0.002 


.0..2 


Student Teaching Staff Ratio 




0.000 


0.000 


Time in Labour Force (months) 




0.000 


-0.000 


First Generation Immigrant 




0.042 


0.046 


P* Gen. Immigrant * PISA Score 




-0.038 


-0.036 


Second Generation Immigrant 




0.003 


0.007 


2"“^ Gen. Immigrant PISA Score 




0.033 


0.034 


Anglophone in Quebec 




0.109* 


0.106* 


Anglophone ’* PISA Score 




0.002 


0.000 


Francophone Outside Quebec 




0.002 


0.001 


Francophone * PISA Score 




0.000 


-0.001 


Hrs. Worked While in High School 




-0.001* 


-0.001* 


HS Hours*PISA Score 




0.000 


0.000 


Hrs. Worked While in PSE 




-0.002* 


-0.003* 


PSE Hrs.*PISA Score 




0.000 


0.000 


Province (ref. group: Ontario) 








NFLD 




0.011 


0.010 


PEI 




-0.036 


-0.036 


NS 




-0.038 


-0.036 


NB 




-0.013 


-0.013 


QUE 




-0.060* 


-0.059* 


MAN 




-0.052* 


-0.051* 


SASK 




-0.052* 


-0.051* 


ALTA 




-0.067** 


-0.064** 


BC 




-0.027 


-0.026 


Educational Attainment (ref. group: < HS) 








HS grad 






0.002 


College Leaver 






0.026 


University Leaver 






-0.004 


College Graduate 






0.010 


University Graduate 






0.203* 


Pseudo R^ 


0.01 


0.04 


0.04 


No. of Obs. 


3809 


3809 


3809 



* indicates significance at 5%, ** indicates significance at 1% 



51. For women, the net effect of a standard deviation increase in the PISA reading score, relative to 
other women, is to reduce the prohahility of unemployment hy approximately 2 percent.'"^ Introducing the 
control variables for family background, school resources and so on reduces the estimated effect, as 
expected, but the estimate is no longer statistically significant. Adding educational attainment indicators 



14 . Note that the values reported in Tables lOA and lOB are the marginal effects, not the coefficient estimates 
of the probit equation. 



29 




EDU/WKP(2010)2 



further reduces the size of the coefficient on the PISA score hut, of course, it remains insignificant. Indeed, 
very few of the control variables achieve statistical significance. 

52. A similar story holds for males. Using only the PISA score in column 1, a standard deviation 
increase in the score reduces the prohahility of unemployment hy 2.5 percent hut, once again, the estimate 
loses its significance once control variables are added. 

53. Among these control variables, the male estimates show a more pronounced provincial pattern in 
the expected way, with unemployment significantly lower in Quebec and the Prairie provinces relative to 
Ontario. These differences do not appear to apply to women. The genders also differ in the estimates of the 
benefits of working while in school. For males, working while in high school and while in PSE (for those 
that go on to PSE) is associated with a reduction in the unemployment probability, albeit a small one. 
Women seem to derive no such benefits from working while studying.'^ 



15 . This may be related to the fact that males are considerably less likely to work while in school but, when 
they do, tend to work much longer hours. See Usalcas and Bowlby (2006). This may more effective in 
building labour market experience. I am grateful to a reviewer for pointing this out. 
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VI. CONCLUSIONS AND RECOMMENDATIONS EOR EURTHER RESEARCH 



54. The primary objective of this paper was to discover whether an individual’s performance in the 
PISA reading tests at the age of 15 has effects on his or her labour market outcomes, in the form of either 
higher earnings or lower unemployment. In contrast to international findings (using other measures of 
cognitive skills among teenagers), test scores are found to have little impact on either outcome once 
account is taken of the students’ backgrounds. There is some evidence that scores matter among women 
with completed high school or less and may be slightly responsible in an indirect way for them through 
higher educational attainment among them. 

55. These results are somewhat surprising given the existing literature on the importance of cognitive 
skills in the labour market but two important caveats about the results must be born in mind. First, an 
econometric examination of the link between PISA reading scores and labour market outcomes is really a 
test of a joint hypothesis: that cognitive skills among teenagers matter upon labour market entry and that 
these skills are accurately measured by the PISA test. It is conceivable that the PISA tests do not elicit 
accurate information about the cognitive skills that may eventually be rewarded in the labour market, either 
because of poorly designed questions or because of the way in which the tests were administered. For 
example, males are notorious for “not showing up” for tests that they perceive to have little immediate 
reward.'® Secondly, respondents, especially those completing post-secondary education, are recent labour 
market entrants and their early experiences may be very poor predictors of their longer term success. In 
particular, the types of skills that result in high PISA scores may well be very difficult for employers to 
measure until some time has passed in the relationship with the employee and the rewards for those skills 
may therefore be some time in coming. If this investigation were to be conducted several years hence, the 
results might be quite different. 

56. On the other hand, it may simply be true that PISA scores are not very important in the longer 
run. The particular skill set measured by them may not be of particular value to employers several years 
later. As well, it may be true that these scores serve only as proxies for parental and school investments in 
children and it is these investments, not the scores per se, that are important. These background factors are 
important determinants of the scores, as shown in Table 1, and once we control for them, there may be 
little more that the PISA outcomes can inform us about an individual’s future labour market success. In 
other words, the correlation between PISA scores and family background may be so highly correlated that 
those scores add very little independent predictive power to an equation that contains measures of that 
background. Finally, we must remember that a significant amount of time elapsed between the 
administration of the PISA tests in 2000 and the observed labour market outcomes at the end of 2005. 
During those years, the YITS cohort A respondents continued to learn through both education and 
experience. Perhaps it is unreasonable to expect that a teenager’s ranking in the ability distribution as 
measured by PISA scores is a perfect predictor of his or her ranking in the ability distribution as measured 
by labour market needs five years later. 



16 . This may explain the gender difference in this paper’s findings. 
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57. If PISA scores do not matter for early labour market outcomes, then why do they matter so much 
for educational attainment?'^ One possible explanation is that PISA tests do not determine the labour 
market readiness of the student or the extent of marketable skills he or she possesses. Rather, they might be 
measuring academic aptitude . . . the ability to do well in school work. If that is true, we would expect the 
impact of higher scores to work almost exclusively through higher educational attainment. Alternatively, 
the potential labour market productivity of high PISA scorers may exist but may take employers some time 
to observe. The scores of these new employees will not produce immediate wage gains since PISA scores 
cannot be interpreted by employers and they may have to wait some time to fully reveal their higher 
productivity. The YITS does not allow us an adequate opportunity yet to observe wage growth but, in 
future cycles, we might expect to see higher rates of growth among those with higher PISA scores. 

58. This paper may have “jumped the gun” in attempting to assess the impact of PISA scores using 
the YITS data. The analysis must be conducted again once sufficient time has elapsed both to increase the 
sample size of YITS respondents who have left formal schooling for the labour market and to allow a 
better read on their longer term success in that market. As pointed out earlier, YITS respondents from the 
higher end of the PISA score distribution are more likely to be excluded in the analysis in this report since 
they are more likely to be university continuers. This early look at outcomes biases the sample to those 
with lower scores. Waiting for larger sample sizes would also, of course, permit an examination of the 
impact of the math and science scores. 

59. An additional recommendation for further research revolves around the findings in Table 1 that 
province of residence while in high school matters for PISA scores. If it is determined that raising these 
scores is a desirable policy objective, some kind of policy lever is required. The only feasible lever is better 
resourcing of primary and secondary schools. Whether this would be effective requires a clearer 
understanding of whether the observed provincial differences can be traced to differences in funding levels 
across provinces. 



17. I am indebted to one of the reviewers for raising this question. 
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APPENDIX 



Table A1. Means and Standard Deviations 



Variable 


Mean 


Std. Dev. 


PISA Reading Score 
Educational Attainment 


536.74 


92.68 


Less than High School 


0.05 




High School Graduate 


0.21 




PSE Leaver 


0.10 




College Graduate 


0.12 




University Graduate 


0.01 




College Student: Continuer 


0.10 




University Student: Continuer 


0.40 




Parent’s Combined Income at Respondent’s Age 15 ($,000’s) 
Mother’s Education 


66.02 


48.70 


Less than completed High School 


0.51 




Some Postsecondary: Incomplete 


0.04 




College 


0.28 




University 
Father’s Education 


0.17 




Less than completed High School 


0.48 




Some Postsecondary: Incomplete 


0.02 




College 


0.25 




University 


0.18 




School’s Physical Infrastructure Index 


-0.19 


0.78 


School’s Educational Resources Index 


-0.10 


1.03 


Student Teaching Staff Ratio 


19.99 


2.93 


Time in Labour Force (months) 


27.43 


15.63 


First Generation Immigrant 


0.04 




Second Generation Immigrant 


0.04 




Anglophone in Quebec 


0.03 




Francophone Outside Quebec 


0.08 




Hrs. Worked While in High School 


9.76 


13.01 


Hrs. Worked While in PSE 

Province of Residence at Time of First Interview 


4.56 


8.81 


NFLD 


0.08 




PEI 


0.06 




NS 


0.10 




NB 


0.09 




QUE 


0.16 




ON 


0.15 




MAN 


0.09 




SASK 


0.10 




ALTA 


0.09 




BC 


0.09 
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